The present disclosure relates generally to surveillance, and more particularly to video matting in surveillance applications.
Image matting is an operation in photo editing that extracts a foreground image from a background region, such that the composite of a foreground image can be imposed onto a background of choice. One technique for image matting includes estimating the foreground opacity or “alpha matte” at every pixel and extracting those pixels that have high foreground opacity. One challenge with this technique, is extracting, with high confidence, initial foreground and background regions that would then guide the matting process in fully determining the foreground opacity at every pixel. To accomplish this, most existing methods rely on manual input that indicates foreground and background regions. For instance, an alpha matte can be estimated efficiently in close form through an elegant formulation of a quadratic cost function.
The use of manual interactions is, however, unsuitable for performing video matting, a process in which the matte estimation of a foreground object from a video sequence is desired. As can be appreciated, video matting may be a more challenging problem when compared to image matting, because manually marking foreground and background regions for every frame of a video sequence comprising a large number of image frames is impractical. One attempt to automate the matting process includes marking keyframes in foreground and background regions, followed by interpolation based on background and optical flow estimation. Another proposed technique to accomplish unsupervised video matting utilizes cues from spectral segmentation. This technique is described by A. Levin., A. Rav-Acha and D. Lischinski in a paper entitled: Spectral Matting, submitted in conjunction with Proc. of IEEE Int'l Conf. on Computer Vision and Pattern Recognition (CVPR), Minneapolis, Minn., June 2007, now published at IEEE Transactions on Pattern Analysis and Machine Intelligence 30(10), 1-14 (2008), which is hereby incorporated by reference. Since it is well known that the image segmentation problem itself is an ill-posed problem, manual interactions are inevitable if one wishes to achieve a reasonable level of accuracy.